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A System and Method For Finding Regions Of Interest For Microscopic Digital 

Montage Imaging 

Field of the Invention 

[0001] The present invention relates to microscopic digital imaging of complete tissue 
sections for medical and research use. In particular it describes a method to find regions 
of interest for high throughput montage imaging of microscope slides using a standard 
microscope and cameras. 

Background of the Invention 

[0002] Laboratories in many biomedical specialties, such as anatomic pathology, 
hematology, and microbiology, examine tissue under a microscope for the presence and 
the nature of disease. In recent years, these laboratories have shown a growing interest in 
microscopic digital imaging as an adjunct to direct visual examination. Digital imaging 
has a number of advantages including the ability to document disease, share findings, 
collaborate (as in telemedicine), and analyze morphologic findings by computer. Though 
numerous studies have shown that digital image quality is acceptable for most clinical 
and research use, some aspects of microscopic digital imaging are limited in application. 
[0003] Perhaps the most important limitation to microscopic digital imaging is a "sub- 
sampling" problem encountered in all single frame images. The sub-sampling problem 

has two components: a field of view problem and a resolution-based problem. The field 
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of view problem occurs when an investigator looking at a single frame cannot determine 
what lies outside the view of an image on a slide. The resolution-based problem occurs 
when the investigator looking at an image is limited to the resolution of the image. The 
investigator cannot "zoom in" for a closer examination or "zoom out" for a bird's eye 
view. Significantly, the field of view and resolution-based problems are inversely 
related. Thus, as one increases magnification to improve resolution, one decreases the 
field of view. For example, as a general rule, increasing magnification by a factor of two 
decreases the field of view by a factor of four. 

[0004] To get around the limitations of single frame imaging, developers have looked at 
two general options. The first option takes the general form of "dynamic-robotic" 
imaging, in which a video camera on the microscope transmits close to real time images 
to the investigator looking at a monitor, while the investigator operates the microscope by 
remote control. Though such systems have been used successfully for telepathology, 
they do not lend themselves to documentation, collaboration, or computer based analysis. 
[0005] The second option being investigated to overcome the limitations inherit in single 
frame imaging is a montage (or "virtual slide") approach. In this method, a robotic 
microscope systematically scans the entire slide, taking an image at every field. The 
individual images are then "knitted" together in a software application to form a very 
large data set with very appealing properties. The robotic microscope can span the entire 
slide area at a resolution limited only by the power of the optical system and camera. 
Software exists to display this data set at any resolution on a computer screen, allowing 
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the user to zoom in, zoom out, and pan around the data set as if using a physical 
microscope. The data set can be stored for documentation, shared over the Internet, or 
analyzed by computer programs. 

[0006] The "virtual slide" option has some limitations, however. One of the limitations is 
file size. For an average tissue section, the data generated at 0.33 um/pixel can be 
between two and five gigabytes uncompressed. In an extreme case, the data generated 
from one slide can be up to thirty-six gigabytes. 

[0007] A much more difficult limitation with the prior systems is an image capture time 
problem. Given an optical primary magnification of twenty and a two-third inch CCD, 
the system field of view is approximately (8.8 mm x 6.6 mm) / 20 = 0.44x0.33 mm. A 
standard microscope slide typically has a specimen area of 25 mm x 50 mm or 12.5 
square centimeters. This requires over eighty-six hundred fields to image this entire 
specimen region. However, the average tissue section for anatomic pathology is 
approximately 2.25 square centimeters. This only requires approximately fifteen hundred 
fields to cover the tissue alone, approximately 80 percent less fields. 
[0008] Traditionally, field rate in montage systems is limited by three factors - camera 
frame rate, image processing speed, and the rate of slide motion between fields. Given 
today's technology, the limiting factor can be reduced to only the camera frame rate. 
Using a 10 frame per second camera for the example above, imaging the entire slide 
would require 860 seconds or 14.33 minutes. If only the region of interest was imaged, 
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this average time could be reduced to 150 seconds or 2.5 minutes; substantially 
increasing the slide throughput of an imaging system. 

[0009] Thus, a system is needed to automatically find the region of interest on a 
microscope slide and image only this region. 

SUMMARY OF THE INVENTION 

[0010] The present invention relates to a method and system for processing a thumbnail 
image from a microscope slide to determine tissue locations on the slide. The system 
comprises an image cropping component, a tissue finding component, and a scan control 
component. The image cropping component crops the thumbnail image and removes 
portions of the image that fall outside of determined slide boundaries. The cropped 
image from the image cropping component is inputted into the tissue finding component. 
The tissue finding component identifies tissue regions by applying a sequence of filters 
that incorporate knowledge of typical appearance and location of tissue and non-tissue 
slide regions. The tissue finding component outputs a tiling matrix whose values indicate 
which tiles should be imaged. The scan control component interprets the tiling matrix 
and transposes positions of the tiling matrix into actual stage coordinate for a microscopic 
imaging. 

[0011] Accordingly, it is an object of the invention to provide a microscopic imaging 
system for whole slide montage in which standard microscope optics, off the shelf 
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cameras and a simple motorized stage can be used to select the region of interest, image 
only this section and produce perfectly aligned image tiles. 

[0012] The present invention uses a pre-scan process applied to a macroscopic image of 
the entire slide, to guide a high-resolution slide scanning process and ensure high-quality 
images of the entire specimen are acquired. The pre-scan process includes an image 
cropping component, a tissue-finding component, and a scan control component. The 
image cropping and tissue finding components identify interesting regions on the slide to 
be scanned. The scan control component generates the control parameters for a 
motorized microscopic imaging system. 

[0013] It is another object of the invention to use a high-resolution slide scanning process 
to control the operation of the motorized stage and camera. This process utilizes 
information gathered by the pre-scan process, namely the imaging regions, to control the 
positioning of the stage to image only the regions of interest and to ensure the individual 
images are well aligned. 

[0014] Additional features and advantages of the invention will be set forth in the 
description that follows, and in part will be apparent from the description, or may be 
learned by practice of the invention. The objectives and advantages of the invention to be 
realized and attained by the microscopic image capture system will be pointed out in the 
written description and claims hereof as well as the appended drawings. 
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Brief Description of the Drawings 

[0015] The accompanying drawings, which are included to provide a further 
understanding of the invention and are incorporated in and constitute a part of this 
specification, illustrate embodiments of the invention that together with the description 
serve to explain the principles of the invention. 

[0016] Fig 1 illustrates an isometric view of the system in a preferred embodiment; 

[0017] Fig. 2 represents the results of the macroscopic image after the cropping 

component has been applied to remove non-slide regions; 

[0018] Fig. 3 represents the results of the find tissue component; and 

[0019] Fig. 4 is an overlay of Fig. 2 and 3 representing the regions of the slide to be 

imaged. 

Description of the preferred embodiments 

[0020] Reference will now be made in detail to the preferred embodiments of the present 
invention, examples of which are illustrated in the accompanying drawings. The 
following paragraphs describe the functionality of the inventive system and method for 
high throughput montage imaging of microscope slides using a standard microscope and 
cameras. 

[0021] Fig. 1 illustrates a preferred embodiment of the invention. In this embodiment, a 
slide 1 12 to be imaged is placed on a thumbnail imaging position in a slide holder on a 
motorized stage 102. A single frame image containing the entire slide is taken with a 
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macro camera 106. This low-resolution image is analyzed by software components to 
determine the locations of tissue on slide 112. This information can then be used to 
generate control parameters for stage 102 and microscopic camera 104 to ensure that the 
scanning process captures high quality images of only the tissue regions, substantially 
reducing the time to scan an average slide. 

[0022] As is obvious to one skilled in the art, although capturing the single macroscopic 
image saves time, it is not necessary for the operation of the invention. Multiple 
macroscopic images may be required to generate control parameters to the accuracy 
required based on the ratio of the macroscopic to microscopic magnifications and the 
camera specifications of each camera, if separate cameras are utilized. 
[0023] Specifically in a preferred embodiment, a pre-scan processing of the low- 
resolution or thumbnail image includes an image cropping component, a tissue-finding 
component and a scan control component. The image cropping component and tissue 
finding component identify tissue regions on the slide to be scanned. The scan control 
component generates the necessary control parameters to scan only the regions of interest 
under the microscopic optics. 

[0024] The first step in processing the thumbnail image consists of flat-field correcting 
the macroscopic thumbnail image using a similar image obtained from the same camera 
and a blank slide. This removes any spatial light anomalies from the thumbnail image, 
which may reduce the efficiency of the tissue-finding component. Given the format, or 
size, of the camera and the aspect ratio of the slide, a portion of the image will contain 
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non-slide objects such as the slide carrier. To remove these features, the thumbnail image 
is cropped to extract only the slide information. 

[0025] The image cropping is accomplished via a two-pass process. The first pass 
determines an approximate location of the slide boundary, and the second pass fine-tunes 
this estimate. The search for the boundary is conducted over upper and lower intervals 
corresponding to the regions expected to contain the upper and lower slide edges, 
respectively. For this discussion, the slide or region of interest is assumed to be 
positioned near the center, vertically, in the thumbnail image. To facilitate this and 
subsequent processing steps, a copy of the thumbnail image is converted to grayscale. 
The portion of the image falling outside of the identified slide boundary is removed. It 
should be noted that the original color image is also cropped at the estimated edge 
locations, and then is uniformly reduced in size to produce a small thumbnail image of 
the slide for rapid, visual slide identification. 

[0026] Since the slide may not be oriented perfectly horizontal in the original thumbnail 
image, the identified slide edges are likely to lie at an angle. Thus, even after cropping, 
there may be remnants of the slide edges or cover slip in the cropped image. Therefore, 
the image-cropping component attempts to identify pixel blocks that likely contain these 
remaining edges and flags these blocks as edges that will not be considered for high 
resolution imaging by the tissue finding component. 

[0027] The resulting cropped grayscale image generated by the image-cropping 
component serves as input to the tissue finding component. This component locates 
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regions in the thumbnail image that contain tissue of interest to a specialist. In order to 
minimize the time and storage space required to accomplish high-resolution slide 
imaging, the inventive system captures only those regions of the slide that contain tissue. 
This approach requires that regions containing tissue be identified in the thumbnail 
image. 

[0028] The tissue finding component identifies tissue regions via a sequence of filters 
that incorporate knowledge of the typical appearance and location of tissue and non- 
tissue slide regions. Initial filtering steps analyze the mean and standard deviation of the 
local pixel intensities. Pixel mean intensities are used to differentiate tissue-containing 
regions from blank and other non-tissue regions, such as those containing the slide label 
or other markings. The standard deviation data represents the amount of variation in 
pixel values and thus is a good indicator of the border between tissue and the blank slide. 
The mean and standard deviation data is combined to generate a threshold value that is 
used to make an initial classification of tissue versus non-tissue. Subsequently, 
morphological filters are applied to refine the classification based on the size and position 
of neighboring groups of potential tissue pixels. 

[0029] The filters which comprise the tissue finding component process the pixels of the 
cropped grayscale thumbnail image in groups that correspond to slide regions, or tiles, 
that can be imaged individually during the high-resolution scanning process. These 
filters ensure that tiles only partially filled with tissue are classified as tissue-containing 
tiles. The final output of the filter sequence is a tiling matrix whose values indicate 
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which tiles should be imaged; the tiling matrix subsequently guides the high-resolution 
scanning process. 

[0030] The above description was based on using the mean and standard deviation of the 
local pixels as the basis for detecting regions of interest. It is obvious to one skill in the 
art that other image characteristics can be also used to identify the specimen from non- 
items of interest such as dust and scratches. 

[0031] This description was also based on processing a gray scale macroscopic image, 
the same processing tools can be applied to each of the color components (traditionally, 
red, green and blue) of a color image. Additional processing tools can also be applied 
between the color components to refine the tissue finding accuracy and to remove 
features such as labels and writing that are not critical to the application. 
[0032] An example of the image cropping and find tissue processing are shown in Fig. 2 
3 and 4. Fig. 2 illustrates the macroscopic image after flat-field correction and image 
cropping. Fig. 3 illustrates the results of the find tissue component. The resulting tile 
matrix shown in Fig. 3 has a one-to-one correspondence to the field of view of the 
microscopic camera. White pixels (binary 1) signify field to be capture and black pixels 
represent regions not to image. Fig. 4 illustrates an overlay Fig. 2 and 3 representing the 
sections of the slide to be imaged. For this application (anatomical pathology), it is 
imperative to image all suspect regions that may contain tissue so conservative criteria 
were used in the find tissue component, resulting in cover slip edges and writing etched 
into the slide to be identified as to be imaged. The savings in the acquisition time is 
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representative by the ratio of the white to black areas of Fig. 3. For this image, only 53% 
of the slide region is to be imaged, including the label and cover slip edges, and etched 
writing on the slide. 

[0033] At the completion of the find tissue component, the scan control component 
interprets the find tissue tile matrix (Fig. 3) and transposes the positions into actual stage 
coordinates for the microscopic imaging. A program running on a host computer controls 
the operation by communicating with a stage controller and microscopic camera 104. 
Actual scanning can occur in any fashion such as by rows or columns, or in a step fashion 
to image neighboring areas. 

[0034] The foregoing description has been directed to specific embodiments of this 
invention. It will be apparent, however, that other variations and modifications may be 
made to the described embodiments, with the attainment of some or all of their 
advantages. Therefore, it is the object of the appended claims to cover all such variations 
and modifications as come within the true spirit and scope of the invention. 
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